The Influence of Dimensions on the Complexity of Computing Decision Trees

A decision tree recursively splits a feature space \mathbb{R}^d and then assigns class labels based on the resulting partition. Decision trees have been part of basic machine-learning toolkit for decades. large body work considers heuristic algorithms that compute from training data, usually aiming to minimize in particular size tree. In contrast, little is known about complexity underlying computational problem computing minimum-size given data. We study this with respect number d dimensions \mathbb{R}^d, which contains n examples. show it can be solved O(n^(2d + 1)) time, but under reasonable complexity-theoretic assumptions not possible achieve f(d) * n^o(d / log d) running time. The solvable (dR)^O(dR) n^(1+o(1)) if there are exactly two classes R an upper bound leaves labeled first class.